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Abstract 

This experiment was designed to determine the relationship 
between secondary reinforcing power, as indicated by a new-learning 
measure, and extinction strength of an instrumental approach-to- 
reward response elicited by S*™. To facilitate comparison between 
conditioned response strength and reinforcing power of ST, two 
variables presumed to have differential effects upon extinction 
of the approach response were manipulated. The test procedure also 
was designed to determine whether a secondary reinforcer would 
strengthen a new response when that response occurred in a stimulus 
situation which had never been associated with primary reward, 

An approach-to-reward response sequence was conditioned to 
distinctive stimulus aspects of a straight-runway training apparatus. 
For half of the Ss in each of three response-condition groups, the 
approach response conditioned to sN was followed by primary reward 
on 60 per cent of the training trials; the remaining Ss received 
100 per cent primary reward during training. There were seven albino 
rats in each of the six groups called for by the design, Each § 
experienced 120 trials in the training apparatus, followed by 70 
test trials. At the beginning of testing, a distinctive test box 
was attached to the runway; a lever-pressing response in this box 
caused a door to open and allowed § to proceed through the designated 
section of the runway and approach the empty food cup. 

Partial primary reward of the response to SN had the customary 
effect upon acquisition strength of the runway responses, in that 
partial facilitation occurred in the later stages of training. 

The difference in terminal acquisition strength between partially- 
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and continuously-rewarded Ss, however, was not significant in any 
of the three response-condition groups, Extinction responding was 
stronger and more durable after partial than after continuous reward. 
The response-condition variable had the predicted effect upon extinc- 
tion performance of continuously-rewarded Ss: (1) a more effortful 
response sequence was significantly less resistant to extinction 
than a less effortful response to ST, and (2) performance was stronger 
when the approach response was conditioned and extinguished under 
more rather than less similar stimulus conditions. The latter effect 
also occurred during extinction after partial reward, but the influence 
of a difference in effortfulness was contrary to predictions, Strength 
of the secondarily reinforced lever-pressing response was significant- 
ly greater after partial than after continuous reward of the response 
to SN during training, and the response-condition variable had the 
predicted effect upon lever-pressing performance of all Ss. Despite 
the similar effects of the treatment variables and similar day-to- 
day changes in strengths of both responses, there was no significant 
correlation between lever-pressing speed and extinction strength 
of the response elicited by S* . There apparently is no adequate 
theoretical explanation for this result, 

The demonstrated capacity of a secondary reinforcer to strengthen 
a new response in a new stimulus situation was the unique contribution 
of this experiment, When the new lever-pressing response in a 
distinctive test box was followed by instrumental behavior conditioned 
to stimuli that had preceded primary reward during training, lever 
pressing increased in strength and performance was maintained over 
70 test trials, 
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Introduction 

Many investigations have shown that a neutral stimulus cs) 
which is repeatedly paired with primary reward (sR) will acquire 
reinforcing properties. Such a stimulus is then referred to as a 
secondary reinforcer ($"); its reinforcing properties are demonstrated 
in a test situation where primary reinforcement does not occur and 
where the occurrence of §* is contingent upon a particular instru- 
mental response, The reinforcing power of the previously neutral 
stimulus is measured in terms of the strength of the response that 
produces §", 

It would be universally agreed that all methods of establishing 
a secondary reinforcer involve conditioning of some response to sN, 
Various studies have demonstrated that several of the same variables 
that determine strength of conditioning also determine strength of an 
Ss": Stronger s*'s have been produced by more frequent pairing of sN 
and sk (Bersh, 1951, Exp. 2), larger amounts of primary reinforcement 
(D'Amato, 1955), and an interstimulus interval closely approximating 
the optimum CS-UCS interval in classical conditioning (Bersh, 1951, 
Exp. 1). There is, however, a difference of opinion as to which basic 
type of conditioning is primarily responsible for the development of 
secondary reinforcing properties. [In all successful training proce- 
dures, the occurrence of sN immediately preceding and/or accompanying 
sR provides a situation in which sN could acquire control over either 
classically-conditioned responses or instrumental approach-to-reward 
behavior, 

Several learning theorists (Hull, 1952; Mowrer, 1960; Seward, 
1956) propose that classical conditioning of a fractional anticipatory 
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goal response to s% accounts for the establishment and effectiveness 
of secondary reinforcers. When sN (CS) is paired with s® (UCS), 
fractional components of the goal response (UCR) become conditioned 
to sN, During training the sN is thus established as a conditioned 
elicitor of a fractional anticipatory goal response (ro). The 
occurrence of Ip produces a distinctive stimulus, Sos and this 
hypothetical f'g-Sg mechanism in some way carries the power to streng- 
then instrumental behavior. According to this interpretation, all 
secondary reinforcement training procedures would be sufficient 
(in that they all provide pairing of SN and sR), but those methods 
whereby SN and SR are both presented by the experimenter would be 
considered superior. These methods most closely resemble the usual 
classical conditioning procedure--the experimenter controls the 
occurrence of gN (CS) and sk (UCS) and therefore has control over 
length of the interstimulus interval. Although secondary reinforcers 
have been established by this method (Bersh, 1951, Exp. 1; Estes, 
1949; Fox & King, 1961), subsequent tests of S’ strength have revealed 
only weak and transitory secondary reinforcement effects, After 
reviewing several secondary reinforcement experiments, Wyckoff 
(1959) concluded: "It is probably no accident that this procedure 
has generally been abandoned, usually in favor of a paradigm in 
which § is first conditioned to make some (instrumental) response 
by the use of primary reinforcement paired with the initially neutral 
stimulus." (p. 69). 

According to the instrumental-conditioning hypothesis (Dinsmoor, 
1952; Schoenfeld, Antonitis, & Bersh, 1950; Zimmerman, 1959), the 


mere pairing of sN and sR is not sufficient; the sN acquires rein- 
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forcing properties only when it is followed by an instrumental response 
which is reinforced by gk presentation. In all successful secondary 
reinforcement training methods, some approach-to-reward response 
invariably follows (or coincides with) SN and immediately precedes 

sk. In Skinner box procedures, onset of a distinct stimulus (e.g., 
flash of light, sound of buzzer) precedes approach to sk, in runway 
experiments, an approach response in the presence of distinctive 
goal-box stimulation procures s®. Failure to establish an S$‘ in 

the Schoenfeld et al. (1950) investigation (in which sN presenta- 

tion was controlled by the experimenter) was attributed to the fact 
that onset of sN slightly followed instigation of eating behavior 

and therefore could not have become a discriminative cue for approach- 
ing the food tray. The authors concluded that the sN must be estab- 
lished as a discriminative stimulus if it is to function as a secon- 
dary reinforcer. Subsequent Skinner box experimenters have presented 


N before delivery of sk and have taken pains to avoid presenting 


s 
sN while S is hovering over the reward-delivery aperture (i.e., to 
insure the occurrence of some primarily rewarded approach response 
following s¥), 

Assuming the necessity for conditioning of an instrumental 
response to gN during establishment of an S‘, a question would 
still exist as to whether secondary reinforcing power depends upon 
occurrence of this response in the test situation, Results of a 
study by Keehn (1962) indicate that strengthening effects of st 
upon new learning are dependent upon occurrence of the response 


previously conditioned to sN, Keehn first trained rat Ss ina 


24-in.-long double-compartment box to establish sN (buzz plus opening 
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of a partition) as a cue for leaving the restraining compartment 
and entering the other compartment to obtain primary (water) reinforce- 
ment. Secondary reinforcement affects were measured by inserting 
a bar into the restraining compartment and arranging conditions 
so that bar pressing was rewarded by S*, Results of this investiga- 
tion showed that new learning was strengthened only when the pre- 
viously-learned instrumental (approach-to-reward) response was 
permitted during testing--when each bar press produced the buzz, 
but Ss were either not allowed to enter the water compartment or 
required to enter by making a new approach response, there was no 
evidence of learning, For those Ss permitted to make the previously- 
learned approach response, response strength (latencies from partition 
opening to touching water tube) remained approximately the same 
throughout testing. Keehn's results show that S* is not effective 
when the response conditioned to it is physically prevented; they 
do not indicate whether there is a functional relationship between 
response strength (when the response is permitted) and reinforcing 
power of Sf, 

Estes (1949) suggests, on the basis of observational evidence, 
that a new response is strengthened by secondary reward only so 
long as the presentation of S™ continues to elicit the conditioned 
approach-to-reward response. Since primary reinforcement is never 
given during the test of s* strength, initiation of testing marks 
the beginning of extinction for the response conditioned to St, 
If secondary reinforcement is dependent upon elicitation of this 
response, then prolonged extinction responding would increase the 
durability of S¥ effects, Furthermore, it is possible that reinforcing 
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power is positively related to strength of the response elicited 

by S*; in this case, strength of the response reinforced by S* and 
extinction strength of the response elicited by S* should vary 
concomitantly throughout testing. In order to determine the relation- 
ship between secondary reinforcing power (i.e., strength of the 
response that produces ST) and extinction strength of the approach- 
to-reward response, an experiment must provide for the occurrence 

of a clearly defined and independently measured response following 
sN (training) and S* (testing). The present experiment was designed 
so that such a comparison could be made. To increase the precision 
of statistical comparisons and to extend the generality of the 
findings, two variables (percentage of primary reward during train- 
ing, and response condition during training and testing) presumed 

to have differential effects upon extinction of the response to 

SN were manipulated. 

An approach-to-reward response sequence in a straight runway 
(17-in. alley, 9-in. goal box) was conditioned to distinctive stimulus 
aspects of the training apparatus (i.e., black color, smooth texture) 
and to an auditory stimulus (2-sec, buzz) which began as § entered 
the goal box. In the test phase of the experiment, a new response 
(lever pressing) in a distinctive test box (grey color, rough texture) 
allowed § to enter the training apparatus; this response "produced" 

S™ (i.e., stimuli which had preceded sN during training) and permitted 
S to make the approach-to-reward response leading to the now-empty 
food cup. Reinforcing power was measured in terms of lever-press 
speeds, which were compared with extinction scores indicating strength 


of the response to Sf, 
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The approach-to-reward response was conditioned to neutral 
stimulation which occurred on every trial and was followed by sR 
either 100 per cent (continuously-rewarded Ss) or 60 per cent 
(partially-rewarded Ss) of the time. A secondary reinforcement 
study by Zimmerman (1959) indicates that partial reward during 
runway training may produce an extremely durable secondary reinforcer 
when reward value is indicated by strength of a new response, 
Zimmerman first established SN (buzz in combination with lifting 
of start box door) as a cue for running to a goal box containing 
food reinforcement; as a test of secondary reinforcing power, a 
lever was inserted into the start box and conditions were arranged 
so that a lever press produced ST and permitted § to run to the 
empty goal box, After partial primary reward of SN (30 continuously 
reinforced trials, followed by 60 trials with a gradually decreasing 
percentage of reward), partial secondary reinforcement (gradually 
decreasing percentages until one response in 20 was rewarded) of 
the new lever pressing response resulted in stable and prolonged 
(10 to 14 90-min. sessions) responding. Strength of the running 
response conditioned to S*, measured in terms of starting times, 
continued at about the same level throughout testing. It should 
be noted, however, that the results of this experiment could have 
been influenced by two conditions other than partial reward of the 
running response during training: (1) In contrast with the usual 
procedure of using the same percentage of reward throughout training, 
Zimmerman instigated a partial reward schedule only after the instru- 
mental response had been established under continuous reinforcement; 


and (2) Zimmerman used a partial secondary reward schedule during 
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testing. In any case, since all Ss were partially reinforced during 
training, no comparisons can be made as to the relative effects 
of continuous and partial primary reward of the response to sN, 

Several investigators, however, have shown that resistance 
to extinction of runway responses is greater after partial reward 
than after continuous reinforcement, In a study by Freides (1957), 

a comparison of equivalent groups showed significantly faster extinc- 
tion running speed in both alley and goal box after partial reward; 
Theios and Polson (1962) found that partial reward of runway behavior 
produced stronger extinction responding in terms of both running 
speed in alley and goal box and number of trials to reach an extinc- 
tion criterion; and Weinstock (1958) reported that rate of decrement 
in running speed was less and extinction asymptote was higher as a 
function of decreasing percentages of reward during acquisition. 

We would predict, therefore, that partial reward of approach responses 
conditioned to s% in the present experiment will produce stronger 
extinction responding throughout secondary reinforcement test sessions, 
If there is a positive relationship between reinforcing power and 
strength of the response elicited by $8", then the reinforcing value 

of s* should be greater for partially-rewarded than for continuously- 
rewarded Ss. 

There were three response-condition groups (each divided into 
partial- and continuous-reward subgroups): Two of the response- 
condition groups learned two successive responses--hurdle- jumping 
in the alley, and running in the goal box. One of these groups 
(Condition I) was permitted to make both alley and goal-box responses 


during testing--i.e., lever pressing preceded and "released" the 
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entire instrumental sequence; a lever-pressing response by Ss in 

the second group (Condition II) gave access directly to the goal 
box--i.e., lever pressing immediately preceded the final segment 

of the training response sequence, Members of the third group 
(Condition III) were placed at the entrance to the goal box during 
training and were tested under the goal-box-only condition, The 
predicted difference between test performance of continuously-rewarded 
and partially-rewarded Ss should hold for all three response condi- 
tions--despite variation in length of the response sequence and in 
similarity between training and testing conditions, extinction running 
speeds of the partially-rewarded Ss should be faster and their 
response to §* should be more durable. 

Two tentative predictions may be made concerning relative 
extinction strength of the response to S* under the three response 
conditions, Conditions I and III differ only in regard to length 
of the approach-to-reward response, with the longer response (Condi- 
tion I) being clearly more effortful. From Hull's (1952) theory 
of extinction, which hypothesizes that magnitude of reactive inhibi- 
tion is an increasing function of effortfulness of a response, 
it follows that a more effortful response will be less resistant 
to extinction. Capehart, Viney, and Hulicka (1958) found a decrease 
in number of extinction responses as a function of effort (5-, 

40-, 70-gm. loading on a Skinner box lever) required to make the 
response, The inhibition theory of extinction and results of the 
Capehart et al, experiment suggest that Ss trained and extinguished 
on the longer, more effortful response sequence (Condition I) will 


show less resistance to extinction than Ss (Condition III) trained 
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and extinguished on the shorter, less effortful response. If secon- 
dary reinforcing power is dependent upon extinction strength of the 
conditioned approach response, there should be a corresponding dif- 
ference in the reinforcing value of s*, 

Subjects under Conditions II and III were tested with the 
goal-box-only condition, i.e., lever pressing "produced" s* (distinc- 
tive goal-box stimuli) and permitted § to make the conditioned goal- 
box running response, These two conditions differed only in that 
training and testing situations are more similar for Condition III 
than for Condition II. Under Condition II, where alley and goal- 
box responses were learned during training, running in the goal 
box was conditioned to a stimulus pattern which undoubtedly included 
(in addition to the external goal-box stimulation experienced by 
both groups) traces of proprioceptive stimuli produced by the hurdle- 
jumping response in the alley. Since the alley response was not 
permitted during testing, thése response-produced stimuli were 
eliminated from the customary goal-box stimulus pattern; as a result, 
amount of stimulus generalization from training to testing should 
be less for Condition II than for Condition III (Hull, 1952), and 
Ss under Condition II should show a larger decrement in strength 
of the conditioned goal-box running response. Results of an investi- 
gation by Fink and Patton (1953) showed a progressive decrease in 
response strength as more and more elements were eliminated from 
the stimulus pattern to which the response had been conditioned, 
Extinction strength of the goal-box running response should there- 
fore be less under Condition II (tested in an altered stimulus 


situation) than under Condition III (trained and tested under similar 
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stimulus conditions); if there is a positive relationship between 
secondary reinforcing power and resistance to extinction of the 
response elicited by g*, the effectiveness of $* should be greater 
under Condition III than under Condition II. 

Use of a factorial design will make it possible to simultaneously 
determine differential effects of the two experimental variables upon 
strength of the Pe pbidebthouonand response conditioned to S*, Simi- 
larly, it will be possible to determine effects of the various treat- 
ments upon acquisition strength of the response reinforced by ST, Sub- 
sequent statistical analyses will then enable us to discover and des- 
cribe the hypothesized functional relationship bwtween secondary rein- 
forcing power and strength of the instrumental response elicited by Ss*. 
At the same time, demonstration of the effectiveness of a secondary 
reinforcer under the unique test conditions of the present study will 
increase the generality of the secondary reinforcement concept, 

Many investigations have shown that stimuli which precede pri- 
mary reward during training will subsequently exert a reinforcing 
effect upon behavior occurring in the same situation. No previous 
study, however, has measured the effectiveness of a secondary rein- 
forcer when the response which produces Ss occurs under stimulus 
conditions differing from those experienced in the training phase 
of the experiment. The present study was planned in part to determine 
whether a secondary reinforcer (distinctive stimulus aspects of the 
training apparatus) will strengthen a new response when that response 
occurs in a new stimulus situation--a test box designed so that 
visual and tactual cues would be distinctively different from stimuli 


experienced during training. 
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Method 

Subjects 

The $s were 42 experimentally naive male albino rats, approxi- 
mately 50 days old at the beginning of the experiment. Taming and 
adaptation to a 23-hr. food deprivation schedule began seven days 
before pretraining. Each day, Ss were placed on a table top in 
groups of six and allowed to explore for 10 min., during which the 
experimenter picked up and handled each § for 1 min. The animals 
were fed 9 gm. of Purina Dog Chow once a day, immediately after 
testing; water was continuously available in the home cage. The 
Ss were housed and tested at the University of Alberta Animal 
Laboratory. 
Apparatus 

The four separate apparatus components, as shown in Figure 1 
(p. 12), were constructed of 1/4-in. plywood; all parts of the appara- 
tus were 2-1/2 in. wide and 6 in. high (inside dimensions), and were 
covered with hinged lids of clear plastic. The entry box was 5 in, 
long--there was a hinged entrance door at one end and a 3-in, high 
by 2-1/2-in. wide exit (no door) at the bottom of the opposite end; 
component B was a l7-in. alley with two 3-in.-high plastic hurdles 
located 2 in. and 8-1/2 in. from the entrance; and the goal box, 
component C, was 9 in. long, A 1/8-in. thick by l-in. high wood 
barrier extended the width of the goal box, 1 in. from the far end, 
forming a 1- by 2-1/2-in. food cup; this barrier served to conceal 
the presence or absence of reward until § traversed the goal box 
and "nosed" the food cup. The walls and floor of the entry box, 


the alley, and the goal box were painted black; side walls were 


11 


_ 
= beoxqai- ore ies 9 SA 
bas qe impy ja bio aynh OF yl 
RS A ROSE eee Ie ch 50 bale 
ni qod eis? s no beset, stow ag ~ynb dose 
ort iBadahe gash (puditen OF meth ochodgeibeih ieunasin Cat 
rfenins off aut I 26 @ dose Kefbaad bie qu betolq ¥: 
roads ylodatbonms .yab a snd. word) god, aikteY 20° nig 
af .9g89 swod ed ot olidsttave ylepounianos aw 36 
Inmick sttediA to yletovinU 989 38 bodaod baw 


{ otugit nl mode an ,2dnenequcs avteriqqs 93: | 
-siaqgs of% to esxeq (le yhoowelq .n}-A\f to bessuesance 
otsw bas ,(atoldhomth obienz) dyith ot 3 how sBbw oak § 
Jab 2 aod xod “dane off .oktastq parerinalte eis 
Hetil .nt-€ ® tem bow snc an toot acne ean a a 
thns sttagqgo ordi To moased sife da re rb-S\L-$ yd 
aolbrud otsanig ee ee 


od fsog oft bas eRe 


boow dgdti .at+l yd dodtray . 


= case , 
Ree: ecoloes 
ea ae | veil 


ntl Aa seuteieneted ; 

ce Axe he ee ae +g | 

Pay: ite oe - 
yt Lai J dieu c 


e. | face 


*, Av aS 


-_ 


7 
er 


7 
| a 
f 


NEES 


—— 


ENTRY BOX COMPONENT A 


COMPONENT B 


COMPONENT C 


Figure 1, Scale drawing of the apparatus (l/s in, = Bein.), 


lined with thin sheets of clear plastic. A black metal guillotine 
door at the entrance to the goal box could be lowered to prevent 
retracing. Companent A (the lever-equipped test box) was 9 in. 
long; it was painted grey and the side walls and floor were covered 
with hardware cloth, Thus, there was a distinct color and texture 
difference between component A and the rest of the apparatus, At 
one end of the box was a hinged entrance door; the lever, located 
on the opposite wall of the compartment, was 1-1/4 in. long and 
projected 3/4 in, into the box, 4 in. above the floor. A 3-in, 
high by 2-1/2-in. wide grey metal exit door was located immediately 
below the lever. 

Apparatus components could be fitted together in the following 
sequences to fulfill training and testing requirements: Training: 
either entry box-component B-component C, or entry box-component C, 
Testing: either component A-component B-component C, or component A- 
component C, [Introduction into the entry box during training 
placed § directly in front of the open exit; no response was required 
in the entry box, and § could immediately enter the next apparatus 
component, At the beginning of testing, the entry box was removed 
from the apparatus and component A substituted in its place; a lever 
press by § activated a relay which caused the sliding metal exit door 
to lower, revealing an opening identical to the exit from the entry 
box and permitting § to enter the next apparatus component, 

Response times in components B and C were obtained by means 
of Standard Electric ,0l-sec. timers connected to photoelectric 
units placed at the entrance to B, at the entrance to C, and 1 in. 


before the end of C (immediately above the barrier that formed one 
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side of the food cup). Light beams passed through infrared filters 
(rendering them invisible to S) and crossed the runway at a height 
of 1-3/4 in. Interruption of successive light beams (a) started 
one timer, (b) stopped the first timer and started a second timer, 
and (c) stopped the second timer, Activation of the photocell at 
the entrance to component C also caused a muffled 6-v. doorbell 
buzzer to sound for 2 sec. Thus, clocks were activated in such a 


way as to record time taken to traverse the 17-in. alley and time 


of approach to the food cup (8 in.) in the goal box. When component 


B was not used, response time in C was obtained by means of the 
appropriate photocells and timer, 

In order to obtain testing response measures, a microswitch 
and timer arrangement was connected so that closing the entrance 
door to component A activated a microswitch, starting a timer which 


stopped when § pressed the lever. This latency measure represented 


atrength of the new, secondarily reinforced lever-pressing response, 


Other response measures during testing were obtained by means of 
the photocell and timer arrangements described above; the buzzer 


always sounded when § entered component C, 


A separate apparatus was employed during pretraining to acquaint 


S with stimuli produced by the lowering of a door similar to the 
exit door of component A. Since door-opening was not experienced 
during regular training, prior adaptation to these novel stimuli 
(a loud "click" and a rapid movement as the door lowered) was con- 
sidered desirable, This pretraining apparatus was 21 in. long by 
2-1/2 in. wide by 4 in. high; it was divided into two compartments 


by a black metal sliding door which could be manually lowered by 
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the experimenter, 

A round wire cage was used to transport § from the housing 
area to the test room and to detain § during between-trial waiting 
periods, The cage was 7-1/4 in, high, 8-1/2 in. in diameter, and 
had a plastic lid. 
Experimental Design 

As illustrated in Table 1 (p. 16), the treatment groups comprise 
a 2x 3 factorial design. There are two levels (100% and 60%) of 
the primary reward variable, Response condition is varied in three 
ways: I--trained on alley (B) and goal-box (C) responses, tested 
with the lever-pressing response in (A) preceding the BC response 
sequence; II--trained on alley (B) and goal-box (C) responses, 
tested with the lever-pressing response in (A) preceding the C res- 
ponse; and I[I--trained on the goal-box (C) response, tested with the 
lever-pressing response in (A) preceding the C response, Seven Ss were 
randomly assigned to each of the six groups called for by the design, 
Procedure 

Pretraining (6 days). Procedures used on the first two days 
of pretraining were designed to acquaint Ss with the training appara- 
tus, On Day 1, pairs of Ss were given 5 min, to explore the entire ap- 
paratus, On Day 2, individual Ss were placed in the goal box for a 
2-min, period, during which each § obtained 10 97-mg. Noyes Rat Food 
pellets from the food cup, 

The next three days were spend in adapting Ss to stimuli 
associated with opening of a door similar to the one to be used 
during testing. On Day 3, Ss in pairs were allowed to explore the 


pretraining apparatus for 5 min, with the door open. On Day 4 


15 


siete anintinidink ii inal iq 
20 (W08 bus {00L) alevel oud ore exorlt .agtasb Katnotos? £8 a 

sould at bolisv at ooldtbaos eandqaail .oldsiasy brevet esmulaq ‘odd 7 

bodeed ,zsaroques (0) sodeLade bos (6) yells ao bemiowi-+E, tayee 

sadoq297 Of a3 yatbecerq (A) at sanoqads gotesony-sovol oils daw ; 

casanogess (D) xod~Isog bas (@) yolin nO bentert-«ih jeonompss 

~ae% 9 a2 gnibosetq (A) at camogess gnteesaq~eevel odd dzhw begee, | 
ola d3by botees ysanoqaor (0) xod-Ss0, oid a0 beainza--TE2 bam yeandg 

o1sW a2 an¥.d .gamoggos D edd grtbsoosy GA) 9) sanogast sakeseuquzevel 
Landaub ot 4d tO} bolisa aquozy xia odd 20 sess 02 bengieasygindbass | 


aysh owt Jerk? grid no beav aembecomt (avo 2) gaguieoager ee 


-niaqqs golaies? odd dtiw 8G Iniaupos paemanittr se Honsc 


Table 1 


Experimental Treatment Groups 


| | Response Condition 
Se 


Percentage of 


: at | LIL 
Primary Reward | | 
BC-ABC BC-AC | C-AC 
100% (c) Group I-c Group II-c | Group III-c 
60% (p) | Group I-p Group II-p | Group III-p 
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and 5, Ss were individually placed in the pretraining apparatus for 
5 min, Food pellets were scattered on the floor of the runway. 
A period began with the sliding door closed; throughout the period 
the experimenter lowered the door whenever S approached it, All 
Ss received at least 16 door-opening experiences, 

On Day 6, each § was given five trials according to the following 
procedure; (1) S put in entry box and traversed that portion of 
the runway to be used for its particular group during training; 
(2) as S entered goal box, buzzer sounded for 2 sec.; (3) goal box 
door closed and 10 sec, were allowed for obtaining and eating one 
97-mg. food pellet; (4) S removed from goal box to a between-trial 
detention cage where it remained for 30 sec, 

Training (to establish neutral stimulation as a secondary 
reinforcer), Each § received 10 trials per day for 12 days. The 
sN (visual and tactual apparatus cues, and a 2=sec. buzz which 
began as § entered the goal box) occurred on every trial. For the 
three 100% subgroups, every occurrence of SN was immediately followed 
by sR (one 97-mg. food pellet). Subjects in the three 60% subgroups 
received s® on six of the ten daily trials; the other 40% of sN 
presentations were not followed by primary reward. Daily sequences 
of rewarded (R) and nonrewarded (N) trials were based upon an estimate 
as to the probability of runs of different lengths when there are 
two sequentially independent events--one (R) occurring 60% of the 
time; the other (N), 40% of the time--and when the length of the 
sequence is ten. Maximum run length for both R and N was two, there 
were 12 possible sequences, and the first and last trial of each 


sequence was rewarded, The sequence to be used for the first session 
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was randomly chosen from the 12 possibilities, the second choice 
was made from the remaining 11 alternatives, etc, All Ss in the 
60% subgroups received the same sequence on the same day. 

Animals trained under Conditions I and II learned an instru- 
mental response sequence comprising a hurdle-jumping response in 
the alley and a running response in the goal box; this sequence 
occurred in the presence of sN and was followed by sR on either 
100% (Groups I-c and II-c) or 60% (Groups I-p and II-p) of the 
trials. Thus, a relatively long and effortful response sequence 
was learned. An individual training trial for these four groups 
consisted of the following events: (1) S put in entry box and 
traversed alley; (2) as S entered goal box, buzzer sounded for 2 
sec.; (3) door closed, confining S in goal box for 10 sec.; (4) 

S removed from goal box and placed in the detention cage for a 30- 
sec, between-trial interval. 

Animals trained under Condition III were required to learn 
only the running response in the goal box; this instrumental response 
occurred in the presence of SN and was primarily reinforced on either 
100% (Group III-c) or 60% (Group III-p) of the trials, Thus, a 
relatively short, less effortful response was learned. Each training 
trial proceeded as follows: (1) § placed in entry box so that its 
head entered the goal box and activated the 2-sec, buzzer; (2) 
door closed, confining $ in goal box for 10 sec.; (3) S removed 
from goal box and placed in detention cage for 30-sec, between- 
trial interval. 


Testing (to determine S™ strength). Following the 120 training 


trials, all primary reward was discontinued and each § received 
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10 test trials per day for 7 days. The lever-equipped test box 
was substituted for the entry box, and the apparatus was arranged 
so that a lever press was immediately followed by lowering of the 
door between the test box and the next apparatus component--either 
B (for Groups I-c and I-p) or C (for Groups II-c, II-p, III-c, 
and III-p). 

An individual trial began when § was placed in the test box, 
A lever press by Ss tested under Condition I allowed § to proceed 
through the alley and enter the empty goal box, As in testing, 
S remained in the goal box for 10 sec, and was then removed to the 
detention cage for a 30-sec, between-trial interval. A lever press 
by Ss tested under Conditions II and III gave access directly to 
the empty goal box where S was confined for 10 sec., then removed 
and detained in the waiting cage for 30 sec. 

Measures of performance, The following performance measures 
were obtained for various groups during training and testing: 
(1) Running time in the alley (17 in.), (2) running time in the 
goal box (time to traverse the 8 in, between entrance and food- 
cup barrier), and (3) lever-press latency (interval between closing 
of test box entrance door and the first lever press). Each raw 
time score was converted to a speed measure by a reciprocal trans- 
formation. Response speeds in the alley and goal box were taken 
as a measure of acquisition strength of the approach-to-reward 
response conditioned to sN. In order to express extinction strength 
of this response during testing in comparable terms for the various 
groups, the following steps were taken: (1) Mean individual speeds 
on Training Day 12 were calculated for that portion of the runway 
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to be used during testing, i.e., mean alley-plus-goal box speed 
for each § in Groups I-c and I-p, mean goal box speeds for Ss in 
the other four groups. (2) Extinction scores for each § were ex- 
pressed as percentages of the mean response speed of that S on 
Training Day 12. The reinforcing power of S* during testing was 


measured in terms of lever-press speeds, 


20 


“a oo wee 


-boaqa xod Inog-aylq-yelia ana ee = Hee ipa obs a 
Ak ag x03 son te Ly 
wee onsine 
-x9 xv 3 ilona tO% aetoor Ao wesc 
bedrest 


ony gmidded gutitub cr anata gains 


.ebsoga avarq-tovel 30 amiss ok bemves ; a 


. 9 


aed 04 = - =? ‘s bs bite se gai & aA my | 7 


’ a gid GQ th olvngi oe >in 
646A @ ‘'s mapnee % [ 
a on! et alae ' 
7% se a 7 i 
a 5 
atin wa -—— sr ete ie 
co * Mteaeal agg’ r ; a 


’ int 
+ eee <are 


Results 
Training 

Mean daily alley and goal box speeds are plotted in Figure 2 
(p. 22). Examination of the curves illustrating alley performance 
of Groups I and II reveals that partially-rewarded Ss ran more slowly 
than continuously-rewarded Ss in the early trials. The curves then 
converge and cross, and partially-rewarded Ss ran faster in the later 
stages of training. This pattern of comparative acquisition respond- 
ing under continuous and partial reward is similar to the results of 
several runway experiments (Capaldi et al., 1962; Goodrich, 1959; 
Weinstock, 1954, 1958), and it has been interpreted in terms of the 
"frustration" hypotheses of Amsel (1958) and of Spence (1960). 

The curves illustrating acquisition of the approach-to-reward 
response in the goal box (Figure 2) do not present a consistent pic- 
ture of comparative partial- and continuous-reward effects. Goal box 
speed was slower in Group I-p than in Group I-c on early acquisition 
trials, whereas partially-rewarded Ss in Groups II and III ran faster 
throughout training. Although all other experiments have found that 
partial facilitation either did not occur at all in the goal box 
(Freides, 1957; Goodrich, 1959, Exp. 1) or occurred later as perfor- 
mance was measured closer to the goal region of the runway (Capaldi 
et al., 1962, Exp. 2; Goodrich, 1959, Exp. 2), there would seem to be 
no theoretical interpretation to adequately explain why partial reward 
effects should vary in this way. A "frustration" theory (Amsel, 1958; 
Spence, 1960) apparently would predict earlier and greater partial 
facilitation in the goal box than in the alley. Such an effect is 


revealed in the present experiment when the alley and goal box 
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Approach speeds as a function of training days. 


The 


two curves illustrate performances of continuous- and partial-reward 


subgroups; each point represents mean performance of 7 Ss on 10 daily 


trials, 
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responses of Group II are compared, and when acquisition under the 
goal-box-only condition (Group III) is compared with alley performance 
of the other two groups. 

Mean response speeds on Training Day 12 were calculated for that 
portion of the runway to be used for each group during testing. Re- 
sults of t-tests revealed that there was no significant difference in 
terminal acquisition strength between partially-rewarded and continu- 
ously-rewarded Ss in any of the three response-condition groups, 
Testing 

Extinction strength of the conditioned approach-to-reward res- 
ponse. Approach-response data (extinction scores) were subjected to a 
trend analysis (Edwards, 1960) in which there were two treatment fac- 
tors (percentage of primary reward, and response condition) and a 
dependent variable consisting of mean response speeds on the seven 
test days. A significant interaction between reward percentage and 
response condition (F = 3.56, df = 2/36, p< .05) suggested that it 
would be advisable to separately analyze the effect of each treatment 
factor at each level of the other factor (Table 2, p. 24). 

Mean overall extinction performance was stronger after partial 
than after continuous reward in all response-condition groups, al- 
though the difference between means is significant only in Group I. 
The significant A x C interactions indicate that the trend of the 
daily means during extinction after partial reward is different from 
the trend after continuous reward at all three levels of the response- 
condition factor, Figure 3 (p. 25) illustrates these trends under 
each of the three response conditions, After approximately equal 


approach speeds on the first test day, continuously-rewarded Ss in all 
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Summary Table; Analyses of Variance of Approach=Response Data 


Source of Variation 


Response Condition [I 


A: Reward percentage 
Error (a) 


C: Days 
A x C: Rew. per. x days 
Error (b) 


Response Condition II 


A: Reward percentage 
Error (a) 


C: Days 
A x C: Rew. per. x days 
Error (b) 


Response Condition III 


A: Reward percentage 
Error (a) 


C: Days 
A x C: Rew. per. x days 
Error (b) 


Continuous Reward (100%) 


B: Response condition 
Error (a) 


C: Days 
B x C;: Resp. cond, x days 
Error (b) 


Partial Reward (60%) 


B: Response condition 
Error (a) 


C: Days 
B x C; Resp. cond, x days 
Error (b) 


Table 2 


Sum of 
Squares 


2.783788 
- 530411 


1, 384064 
- 968400 
0975307 


- 243303 
1.188442 


431422 
- 258313 
994242 


- 604058 
2.079538 


1,188617 
1.064437 
2.717396 


- 687539 
1.933700 


1,675281 
1, 393269 
3.143503 


1.158872 
. 705817 


1.400141 
- 824781 
2.804095 


Mean 
d.f. Square F 
1 2.783788 62,98 
12 -044201 
6 -230677 17.03 
6 -161400 11.92 
72 - 013546 
L -243303 2.45 
iz - 099037 
6 -971904 5,21 
6 - 043052 3712 
72 .013809 
1 - 604058 3.49 
12 ~ 173295 
6 .- 198103 5,20 
6 -177406 4,70 
iz .037742 
2 - 343769 3.20 
18 - 107428 
6 279214 9,91 
12 7136106 4,12 
108 028181 
2 -579436 14.77 
18 039212 
6 #233357 8.99 
12 .- 068732 a. 05 
108 025964 
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Figure 3, Extinction scores as a function of test days. The two 
curves for each response-condition group illustrate performances of 
continuous- and partial-reward subgroups; the three curves for each 
reward-percentage condition illustrate performances of the three 
response-condition groups, Each point represents mean performance of 
7 Ss on 10 daily trials. 
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groups showed weaker maximum performance and an earlier and more 
rapid decline in response strength, Stronger and more durable ex- 
tinction responding after partial reward was predicted and is in ac- 
cord with "frustration" theories of extinction (Amsel, 1958; Spence, 
1960). 

Differential changes in running speed are also indicated by sig- 
nificant B x C interactions in individual analyses of the effects of 
response condition within each level of the reward-percentage factor 
(Table 2, p. 24). Among continuously-rewarded Ss, the decrement in 
running speed began earliest and progressed most rapidly in Group I; 
curves for Groups II and III are more similar in form, with Group III 
showing consistently superior performance and higher maximum strength 
(Figure 3, p. 25). When mean overall extinction performances of the 
three groups were compared by Duncan's multiple range test (see Ed- 
wards, 1960, pp. 136-140), using a protection level of 98 per cent, 
Group III ran significantly faster than Group II and Group I. Overall 
performance was stronger in Group II than in Group I, but the dif- 
ference between means was not significant (~ > .05). The three 
response-condition curves for partially-rewarded Ss (Figure Dee aS ) 
also are reliably different in form: the initial increase in response 
strength persists longer in Group III (50 trials) than in Group I (30 
trials); and Ss in Group II, after exhibiting the largest decrement 
from the last day of acquisition to the first test day, showed pro- 
gressively increasing response strength throughout 60 trials and only 
a slight decrease on the last test day, Comparisons between mean 
overall performances of the various groups (Duncan's test, protection 


level of 98 per cent) revealed that Group I and Group III ran 
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Significantly faster than Group II. Although mean performance was 
stronger in Group I than in Group III, the difference between means 
was not significant ( >.05), 

Thus, the response-condition variable had the predicted effect 
upon overall extinction performance of Ss trained under continuous 
reward, The more effortful response sequence (Group I-c) was less 
resistant to extinction than the shorter, less effortful response to 
gf (Group III-c); and extinction responding was stronger when the ap- 
proach response was conditioned and extinguished under more (Group 
III-c) rather than less (Group II-c) similar stimulus conditions. The 
latter effect also occurred during extinction after partial reward-- 
overall extinction performance of Ss both trained and tested in the 
goal box (Group III-p) was stronger than the performance of Ss trained 
on the alley-plus-goal box sequence and shifted to goal-box-only at 
the beginning of extinction (Group II-p). For partially-rewarded Ss, 
however, the effect upon extinction performance of a difference in 
effortfulness was contrary to the prediction that the more effortful 
response (Group I-p) would show less resistance to extinction. Law- 
rence and Festinger (1962, pp. 143-152) report a similar phenomenon 
and suggest that increased effort during acquisition and extinction 
intensifies the partial reward effect. In an experiment in which 
both percentage of reward and amount of effort were varied, resistance 
to extinction after partial reward was markedly greater under a high- 
effort than under a low-effort condition, 

Examination of Figure 3 (p. 25) reveals that the curves do not 
resemble usual runway extinction functions. In most cases (e.g., 


Freides, 1957; Theios & Polson, 1962; Weinstock, 1954, 1958), response 
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Strength begins to decrease after the first few nonrewarded trials 

and becomes progressively weaker throughout extinction, Some investi- 
gators, however, have observed that the characteristic decrease in 
response strength during extinction was preceded by a pronounced in- 
crease in vigor on early extinction trials. This period of enhanced 
response strength has been attributed to the effects of frustration 
produced by nonreward (see Brown, 1961, Ch. 6 for a discussion of 
relevant experiments and theoretical interpretations), Although run- 
ning speeds increased before they decreased in all groups in the 
present experiment, this increase occurred only after an initial, 
rather severe decrement on the first test day. This marked initial 
decrease in response strength could be attributed to the unique ex- 
tinction conditions of this experiment: as a consequence of the test 
procedure, generalization decrement occurred when the stimulus complex 
to which the approach response had been conditioned was altered by the 
addition of stimuli produced by the lever-pressing response in the 
test box, If the initial decrement can be explained in this way, then 
subsequent trends can be interpreted in terms of presumed motivational 
and associative properties of frustration. 

Strength of new response reinforced by sé, Results of a trend 
analysis (Edwards, 1960) of the lever-press data are summarized in 
Table 3 (p. 29). In this case, there was no significant interaction 
between reward percentage and response condition, Average speed of 
lever pressing under all three response conditions was significantly 
faster in the partial- than in the continuous-reward subgroups. This 
finding is similar to the results of several investigations (D' Amato 


et al., 1958; Klein, 1959; Saltzman, 1949) in which strength of a 
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Table 3 


Summary Table: Analysis of Variance of Lever-Press Data 


Sum of 

Source Of Variation Squares 

A: Reward percentage 181,955 

B: Response condition 801,285 

A x B: Rew. per. x resp. 63,232 

cond, 

Error (a) 1,147,357 

C: Days 722,830 

A x C: Rew. per. x days 68,563 

B x C: Resp. cond, x days 349 , 414 

A x Bx C: Rew. per. x resp, 63,374 
cond, x days 

Error (b) 1,933,802 


Coke 


Mean 
Square 


181,955 
400,643 
31, 616 


31,871 
128,805 
L427 
29,118 
5,281 


9,953 
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secondarily reinforced response was relatively greater after partial 
than after continuous primary reward of the response to sN during 
training, The response-condition effect also was significant; 
individual comparisons between means (Duncan's test, protection 
level of 98 per cent) revealed that speed of lever pressing after 
both partial and continuous reward was significantly faster in 

Group III than in Group II and Group I, Although Group II perfor- 
mance was stronger than Group I, the difference between means was 
not significant (“>,05). 


Figure 4 (p. 31) illustrates speed of lever pressing in the 


various groups as a function of test days. To facilitate comparisons 


between these trends and curves portraying extinction strength of 


the approach-to-reward response, curves are plotted separately for 


each response condition and for each reward percentage, A reinforcing 


effect of S™ upon the new lever-pressing response is evidenced by 
an increase in response strength in all groups. Speed of lever 
pressing at first became progressively faster as a function of 
trials (ranging from 20 in Group I-c to 50 in Group I-p), and then 
began to decrease, Mean response strength, however, was signifi- 
cantly (p <.01) greater on the last than on the first test day in 
all groups, indicating that S™ continued to exert a reinforcing 
effect upon the lever-pressing response over at least 70 trials. 
The nonsignificant A x C interaction (Table 3, p. 29) shows 
that the trends of the daily means are not reliably different after 
partial- and continuous-reward training, In all groups, response 
strength on the first test day was approximately equal and the 


eventual decline occurred at about the same rate under both reward 
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Figure 4, Speed of lever pressing as a function of test days, 
The two curves for each response-condition group illustrate perfor- 
mances Of continuous- and partial-reward subgroups; the three curves 
for each reward-percentage condition illustrate performances of the 
three response-condition groups. Each point represents mean perfor- 


mance of 7 Ss on 10 daily trials. 
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conditions; thus, the overall difference between continuous and 
partial performance apparently was related to maximum strength of 
s* rather than to differential loss of reinforcing power, Taking 
into consideration the nonsignificance of the triple interaction, 
the significant B x C effect indicates that forms of the three 
response-condition curves are reliably different at both levels of 
the reward-percentage factor, Under both reward conditions, rising 
and falling slopes are markedly peuhans in Group III than in the 
other two groups. Again, differential overall response strength 
(LID>II>I) appears most directly related to maximum reinforcing 


power of st, 


Relationship between secondary reinforcing power and resistance 
to extinction of the approach-to-reward response. [In general, results 


of the trend analyses suggest some positive relationship between 
speed of lever pressing and strength of the response elicited by 
S®, Overall strength of both responses was greater after partial 
than after continuous primary reward of the response to sN during 
training. For continuously-rewarded Ss, the response-condition 
variable had similar effects upon overall lever-pressing speed and 
overall extinction strength of the approach response (III>II-I). 
Comparison of the curves in Figures 3 and 4 reveals that strengths 
of both responses tended to increase and decrease in a similar 
fashion as a function of test days. 

As a direct measure of the relationship between secondary 
reinforcing power and extinction strength of the conditioned approach- 
to-reward response, the mean daily lever-pressing speed and the 


mean daily extinction score of each § was paired and a coefficient 
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of correlation was computed. 


The positive correlation between 


the two performance measures was low and nonsignificant (r = .03, 


df = 292, p >.10--one-tailed test of significance), 
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Discussion 

The most noteworthy result of this experiment is the demon- 
strated effectiveness of a secondary reinforcer in a stimulus situa- 
tion which had never been associated with primary reward. In all 
previous investigations in which a new response was strengthened 
by S™, the reference response occurred in the presence of stimuli 
which had preceded primary reward during training. There would 
be reason to suspect that these stimuli might have played a crucial 
part in the secondary reinforcement process. 

Acceptance of the hypothesis that conditioning of an instru- 
mental response to sN is necessary for the establishment of secondary 
rewards would not deny concurrent classical conditioning of fractional 
anticipatory responses to stimuli in the training situation, [In 
fact, some learning theorists (Logan, 1960; Mower, 1960; Spence, 
1960) suggest that strengthening of all instrumental behavior is 
dependent upon classical conditioning of anticipatory responses to 
various stimuli in an instrumental learning situation. Spence (1960), 
for example, hypothesizes a learned motivational factor (K) based 
upon internal stimulation (so) produced by a fractional anticipatory 
goal response (rg) conditioned to external stimuli preceding primary 
reward, and suggests that reward variables affect instrumental 
performance by virtue of their effect upon strength of the rpg-Se 
mechanism, Thus, Spence proposes an anticipatory response mechanism 
that strengthens instrumental behavior through producing an increase 
in general drive level. Other theorists (Mowrer, 1960; Osgood, 

1953) believe that some stimuli in an instrumental learning situa- 
tion become conditioned elicitors of anticipatory responses that 
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produce a learned motivational state, while other aspects of the 
situation acquire drive-reducing (reinforcing) properties. 
Specifically, it is possible that "incidental" stimulus aspects 
of the training situation acquire the capacity to arouse a secondary 
drive which is reduced by occurrence of the stimulus event (sN) 
immediately and consistently preceding primary reward, and that 
effectiveness of an S$ during testing may depend upon continuation 
of this motivational pattern, For example, in runway training pro- 
cedures used to establish goal box stimulation as a secondary rein- 
forcer (e.g., D'Amato et al., 1958; Klein, 1959; Saltzman, 1949), 
anticipatory responses may have been conditioned to cues arising 
from the alley preceding the goal box as well as to the designated 
sN (distinctive goal box stimulation), In subsequent tests of 
secondary reinforcement effects, the goal box was removed from the 
end of the alley and replaced by the stem of either a T- or U-maze; 
the previously rewarded goal box was attached to one arm of the 
maze, a neutral box to the other arm; and S* strength was measured 
in terms of "correct" choices (i.e., number of trials on which the 
previously rewarded goal box was chosen). The above studies found 
greater S‘ strength after partial than after continuous primary 
reward and attributed differential effects upon the new (choice) 
response to the secondary reward values acquired by the distinctive 
goal box stimuli. It should be noted, however, that the new response 
occurred in the presence of alley cues, An extension of Mowrer's 
(1960) theory that a "secondary reinforcer is, basically, a stimulus 
which mediates a reduction in (secondary) drive" (p. 265) might 
suggest that secondary reward effects are dependent upon both motiva- 
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tional (drive-increasing) and reinforcing (drive-reducing) proper- 
ties acquired by various stimuli in the training situation. When 
this suggestion is applied to the maze-learning secondary reinforce- 
ment procedure, it is possible that stimuli prior to the reference 
(choice) response aroused a learned motivational state which was 
reduced when a correct choice produced S* (stimuli arising from 

the previously rewarded goal box). Similarly, it is possible that 
stimuli arising from the restraining compartment in the Keehn (1962) 
and Zimmerman (1959) investigations aroused a secondary drive which 
was reduced when the new lever-pressing response produced St--i.,e., 
stimuli which had immediately preceded SN during training. 

Thus, the generality of the secondary reinforcement phenomenon 
is extended by this demonstration of S* effectiveness when the stimuli 
in the test situation could not have acquired drive-inducing pro- 
perties during training. This finding indicates that the reinforcing 
function of an S™ is not dependent upon reduction of secondary 
drive, unless one should wish to propose the arousal of a learned 
motivational state by stimuli which preceded introduction into the 
apparatus during training and testing--a not unreasonable sugges- 
tion, incidentally, when one considers such consistent events as 
pre-session removal from home cage, transportation to the experi- 
mental apparatus, and between-trial detention in the waiting cage. 
Nevertheless, strengthening of lever pressing in this experiment 
did provide a unique demonstration of the reinforcing effect of 
an S‘ upon a new response in a new test situation, 

It is important to note, however, that the St produced by 


each lever press in the distinctively different test box elicited 
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an instrumental response that carried § out of the new situation 

and into the presence of stimuli that had preceded primary reward 
during training. Zimmerman (1959) accounted for secondary reinforce- 
ment effects in his experiment in terms of a Guthrian contiguity 
theory, and attributed strengthening of lever pressing to the fact 
that the response to S* immediately removed the animal from the 
restraining compartment and assurred that the lever-pressing response 
was the last to occur in the presence of these stimuli. [It is 
possible that these particular test circumstances were at least 
partly responsible for the extreme durability of secondary reinforc- 
ing power in the Zimmerman experiment, After partial primary reward 
and under a variable-ratio secondary reinforcement schedule, indi- 
vidual animals made thousands of lever-pressing responses during 

10 to 14 90-min. sessions. In Keehn's (1962) investigation, in 
which the response to sN was continuously rewarded during training 
and in which the S* immediately following each lever press served 

a similar "removal" function, strength of a secondarily reinforced 
lever-pressing response increased as a function of trials one through 
seven and was then maintained at about the same level throughout 

40 test trials. 

Durability of a new, secondarily reinforced response in both 
of these experiments was much greater than in Skinner box studies 
(e.g., Bersh, 1951, Exp. 1; Estes, 1949, Exp. 1; Fox & King, 1961) 
where, according to a contiguity interpretation, the response elicited 
by S* did not maintain strength of the new response by removing 
S from the presence of the stimuli associated with lever pressing. 


It is thus entirely possible that the "removal" function of S* was 
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a necessary part of the secondary reinforcement process under the 
test conditions of the present experiment, In any case, it would 
not be legitimate to generalize from these results and conclude 
that an S™ would be effective in a new situation under other test 
circumstances, 

There was no direct evidence to support the proposition that 
secondary reinforcing power is dependent upon extinction strength 
of the response elicited by S*, When the curves in Figures 3 and 
4 are compared, the lever-pressing and approach-response trends for 
each group appear roughly similar (in that they both increase and 
then decrease), but there is no point-for-point correspondence between 
variations in daily means, The direct measure of the purported rela- 
tionship showed no significant correlation between individual daily 
speeds of lever pressing and speeds of the conditioned approach- 
to-reward response, This result is difficult to reconcile with the 
fact that the treatment variables did tend to have the same general 
effects upon the lever-pressing response and upon extinction of the 
response elicited by ST, For both responses, maximum strength was 
greater and overall test performance was stronger after partial than 
after continuous reward during training; and, for continuously- 
rewarded Ss, the response-condition variable had the same effect 
upon overall strength of the two responses, 

A theoretical interpretation based upon a classically conditioned 
anticipatory response mechanism would explain effects of the treatment 
variables upon both the lever-pressing and the approach response in 
terms of the effect of these variables upon extinction of a fractional 


anticipatory goal reaction, If one attributes acquired reinforcing 
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power to elicitation of an anticipatory response (Hull, 1952; Mowrer, 
1960; Seward, 1956), then secondary reinforcing power would depend 
upon strength of the anticipatory response and would decrease as 

this response underwent extinction. According to several theorists 
who hypothesize an To-So mechanism (Logan, 1960; Spence, 1960), 
extinction of all instrumental behavior is related in exactly the 
Same way to extinction strength of a conditioned anticipatory response, 
Thus, strength of the secondarily reinforced response and extinction 
strength of the approach-to-reward behavior should vary concomitantly 
as a function of the same learned interval reaction. Since there 

was no direct relationship between strengths of the two responses 

in the present experiment, effects of the treatment variables cannot 
be interpreted in terms of their influence upon extinction of an 
underlying anticipatory-response mechanism, 

The fact that there appears to be no adequate theoretical ex- 
planation for this aspect of the results does not detract from the 
interest of these and other findings. Effects of the reward-percentage 
variable upon acquisition and extinction of the response to SN indicated 
that, in general, partial reward had its characteristic influence 
upon instrumental response strength. The finding of earlier and 
stronger partial facilitation in the goal box than in the alley 
during acquisition, although in contrast with the results of previous 
research, would seem to be in accord with a "frustration" interpreta- 
tion of partial-reward effects, The nature of the interaction between 
reward-percentage and response-condition variables suggests that 
the differential effect of partial vs. continuous reward upon resistance 


to extinction is greater when a long and more effortful, rather 
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than a short, less effortful response sequence is involved, Of 
greatest interest, perhaps, was the effect of the reward-percentage 
variable upon secondary reinforcing power under the unique conditions 
of this experiment. Generality of the partial-reward phenomenon 

is increased by the demonstration that an S* associated with partial 
reward during training has a stronger reinforcing effect upon a new 
response established in a new stimulus situation. Although it is 
usually assumed that the influence of training reinforcement condition 
upon secondary reinforcing power is conveyed by its effect upon 
extinction of the response (either instrumental or internal) elicited 
by S™, there apparently was in this case no direct correlation between 
strength of the secondarily reinforced response and of the conditioned 


approach-to-reward behavior, 
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Summary 

This experiment was designed to determine the relationship 
between secondary reinforcing power, as indicated by a new-learning 
measure, and extinction strength of an instrumental approach-to- 
reward response elicited by S*, Training procedures provided for 
the establishment of a clearly defined approach-to-reward response 
in the presence of sN, and test conditions were arranged so that 
a new lever-pressing response in a distinctive test box "produced" 
S* and permitted § to make the conditioned approach response. 

To facilitate comparison between conditioned response strength 

and reinforcing power of §*, two variables presumed to have dif- 
ferential effects upon extinction of the response elicited by st 
were manipulated, The test procedure also was designed to determine 
whether a secondary reinforcer would strengthen a new response 

when that response occurred in a stimulus situation which had never 
been associated with primary reward. 

An approach-to-reward response sequence was conditioned to 
distinctive stimulus aspects of a straight-runway training apparatus: 
one response-condition group was trained on an alley and goal-box 
response sequence and was permitted to make both responses during 
testing; a second response-condition group was trained on an alley 
and goal-box response sequence but permitted to make only the goal- 
box response during testing; and a third group was both trained 
and tested on the goal-box response. For half of the Ss in each 
response-condition group, the approach response conditioned to sN 
during training was followed by primary reward on 60 per cent of 


the trials; the remaining Ss received 100 per cent primary reward 
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during training. There were seven albino rats in each of the six 
groups called for by the design, Each S experienced 120 trials in 
the training apparatus, followed by 70 trials during the test phase 
of the experiment. At the beginning of testing, a distinctive 

test box was attached to the runway; each lever-pressing response 
in this box caused the exit door to open and allowed S to proceed 
through the designated section of the runway and approach the empty 
food cup. 

Partial primary reward of the response to SN had the customary 
effect upon acquisition strength of the alley response. After 
comparatively weaker performance on early trials, partially-rewarded 
Ss ran faster in the later stages of training. Partial facilitation 
tended to be greater and to occur earlier in the goal box than in 
the alley, a result which is not consistent with findings of other 
investigators but which apparently would be in accord with a "frustra- 
tion" interpretation of partial-reward effects. The difference 
in terminal acquisition strength between partially- and continuously- 
rewarded Ss was not significant in any of the three response-condition 
groups, Extinction responding was stronger and more durable after 
partial than after continuous reward. The response-condition variable 
had the predicted effect upon extinction performance of continuously- 
rewarded Ss: (1) a more effortful response sequence was significantly 
less resistant to extinction than a less effortful response to Ss", 
and (2) performance was stronger when the approach response was 
conditioned and extinguished under more rather than less similar 
stimulus conditions, The latter effect also occurred during extinction 


after partial reward, but the influence of a difference in effort- 
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fulness was contrary to predictions. Strength of the secondarily 
reinforced lever-pressing response was significantly greater after 
partial than after continuous reward of the response to SN during 
training, and the response-condition variable had the predicted 
effect upon lever-pressing performance of all Ss. Despite the 
similar effects of the treatment variables and similar day-to-day 
changes in strengths of both responses, there was no significant 
correlation between lever-pressing speed and extinction strength 
of the response elicited by S*. There apparently is no adequate 
theoretical explanation for this result, 

The demonstrated capacity of a secondary reinforcer to strenghten 
a new response in a new stimulus situation was the unique contribution 
of this experiment. When the new lever-pressing response in a 
distinctive test box was followed by instrumental behavior conditioned 
to stimuli that had preceded primary reward during training, lever 
pressing increased in strength and performance was maintained over 


70 test trials. 
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